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Introduction and summary 


There has been a sea change in teacher evaluation over the past eight years. 
Inspired in part by President Barack Obama’s policies; schools have instituted 
teacher evaluation systems that include multiple measures of teacher impact. 
Model systems are aligned to systems of continuous improvement, helping 
teachers identify areas of weakness in their practice and linking them with related 
support. This shift toward more formal systems of evaluation is essential to ensure 
high-quality teaching and learning. Evaluation systems are not the only lever for 
improving teacher quality, but when they are well- designed, they can be a critical 
part of teacher development and support because they provide a framework from 
which teachers can improve their practice. 

In recent years, teacher evaluation systems have come under fire in some com- 
munities. Teachers and advocates have argued that student test scores are not an 
accurate or fair way to assess teacher performance. Though only a small fraction 
of the teacher workforce has standardized testing connected to their performance 
evaluation, this argument has taken hold. Nevertheless, many teachers and system 
leaders have embraced the need to improve teacher evaluation systems so that 
they become tools for improving practice and ensuring teachers are receiving 
appropriate supports. 

As federal policies shift to provide states and districts greater flexibility to craft their 
own evaluation systems, Massachusetts offers an interesting model. It has been less 
controversial because test scores serve as merely a check on the system rather than a 
driver of it. In addition, instead of using an algorithm to determine teacher effective- 
ness, Massachusetts empowers school leaders to use their judgment to make these 
decisions. By empowering evaluators and educators — who are able to determine 
their own growth plans if they are high-performing — and embedding the evalua- 
tion system within a broader system of feedback and professional development, the 
Massachusetts model supports continuous improvement of educators. 
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Teacher testimonial 


Lisa Caponigro is a fifth-grade math teacher in Revere, Massachu- 
setts. As a 1 2-year veteran, Lisa's career has spanned the evolution of 
educator evaluation that has taken place in Massachusetts and across 
the nation. Lisa remembers annual evaluations in the old system: the 
infrequent formal observations, the checklists, and the summary of 
findings she would receive at the end of the year. This system was 
more of a pronouncement on the quality of her teaching, rather than 
clear direction for improvement. 1 

Now, under the Massachusetts Educator Evaluation Framework, Lisa 
is evaluated by her principal — a trusted advisor with whom she"talks 
shop" every day and someone who regularly visits her classroom. 
When the Board of Elementary and Secondary Education in Massa- 
chusetts passed new regulations in 201 1 defining a new framework 
for evaluating teachers and administrators, Lisa's district signed on 
as an early implementer. A group of teacher leaders, including Lisa, 


supported initial implementation by supporting their colleagues in 
understanding the evaluation process, writing goals, and receiving 
and using feedback. 2 

Lisa says Revere's new evaluation system creates many new opportuni- 
ties for teachers to reflect on their practice. She explains: "Our old sys- 
tem wasn't growth-oriented. Our current system is all about a continu- 
ous dialogue between my evaluator and me. He visits my classroom 
often and provides feedback that helps me adjust my practice over 
time. I have confidence that he has the full picture of me as a teacher." 3 

Lisa believes the evaluation process is helping her students as well. The 
goal-setting process has helped her to stay on track in her support of 
students who are struggling. Through specific action steps spelled out 
at the beginning of the evaluation cycle, she is able to hold herself ac- 
countable for doing what she believes is necessary to attain her goal. 4 
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The past and present of 
educator evaluation nationwide 


Prior to 2012, teacher evaluation in the United States was sporadic and uniformly 
viewed as unproductive. Most states and districts failed to make meaningful distinc- 
tions between their best teachers and others. A 2009 study found that, even in dis- 
tricts that utilized a broad range of rating measures, 94 percent of teachers received 
one of the top two ratings, and fewer than 1 percent were rated unsatisfactory. Many 
teachers did not receive feedback tied to their performance, and those who did often 
received feedback only once per year. This feedback typically was not accompa- 
nied by actionable steps to help teachers improve their instruction, and it was not 
used to determine personnel decisions, such as attaining tenure or receiving a raise. 
Among teachers who were surveyed in 2009, 73 percent said that their most recent 
evaluation did not identify any areas in which they needed to develop skills. And 63 
percent of administrators said their district was not doing enough to identify, com- 
pensate, promote, and retain the most effective teachers. 5 

Enter Race to the Top, or RTTT, and the U.S. Department of Educations waiv- 
ers from the No Child Left Behind Act, or NCLB. The RTTT initiative encour- 
aged states to conduct annual teacher evaluations based on multiple measures of 
instructional performance, including student achievement results. Additionally, 
states received points for using multiple rating categories to classify teacher per- 
formance and for using evaluation results to drive professional development and 
inform decisions regarding teacher career advancement. 6 Congress appropriated 
approximately $5 billion for Race to the Top between 2009 and 2012, qualify- 
ing RTTT as the biggest competitive grant program ever administered by the 
Department of Education. 7 

In place since 2012, the Department of Educations NCLB waiver policy required 
all states that received a waiver to “develop, adopt, pilot, and implement, with the 
involvement of teachers and principals, teacher and principal evaluation and sup- 
port systems.” 8 The department required these systems to be used for continuous 
improvement of instruction; meaningful differentiation of performance; regular 
evaluation of teachers and principals; provision of clear, timely, and useful feed- 
back; and informed personnel decisions. 9 States were to use multiple and valid 
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measures in determining educator performance levels, including data on student 
growth for all students as a significant factor. 10 Subsequently, the department 
allowed states to postpone the use of student growth data based on state assess- 
ments to allow for a smooth transition to more rigorous assessments that aligned 
with Common Core State Standards. 11 

NCLB waivers also required that these evaluation systems be used for more than 
providing feedback for teachers to improve their instruction. States were to use 
the evaluation system to review the quality of all staff of schools that were identi- 
fied as within the bottom 5 percent of all schools in their state — based on achieve- 
ment and lack of academic progress — and high schools with a graduation rate of 
less than 60 percent over three years. They were to retain only those staff whom 
the evaluation determined to be effective and those who had the ability to be suc- 
cessful in the turnaround effort. States that sought waivers were required to create 
evaluation systems designed to prevent ineffective teachers from transferring to 
low-performing schools. Additionally, these states were required to provide job- 
embedded, ongoing professional development informed by teacher evaluations, as 
well as support systems tied to teacher and student needs. 12 

As a result of this policy, more than two-thirds of states have made consider- 
able changes to their teacher evaluation systems since 2009. 13 Today, 27 states 
require annual evaluations for all teachers, while 45 states require annual 
evaluations for all new probationary teachers. 14 In 2009, by contrast, only 15 
states required all teachers to undergo an annual evaluation. 15 Many states, 
including Delaware and Connecticut, have also adopted improved professional 
development systems and expanded the tools used for measuring instructional 
effectiveness. 16 Nearly every state has revamped how classroom observations 
are conducted: In most states, trained evaluators observe teachers multiple 
times a year, and the evaluators use a research-based rubric to measure teacher 
effectiveness. 17 In 2013, 41 states required or recommended that teachers be 
evaluated on multiple measures as a more complete and accurate reflection of 
performance, while more than half of all states used evaluation systems to target 
professional development opportunities for individual teachers. 18 

In addition to student achievement and classroom observations, many teacher 
evaluation systems now include student and parent surveys, lesson plan reviews, 
teacher self-reflections, and so-called student artifacts. Each of these components 
is designed to provide a more comprehensive and accurate picture of a teacher’s 
performance in the classroom. 19 In 2013, 47 states required or recommended 
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that stakeholders, including teachers, provide input into the designs of new 
evaluation systems. 20 Such new designs would help create systems of continuous 
improvement that support teachers and will hopefully continue regardless of 
federal policy or incentives. 

Additionally, there has been a culture shift in education: Teachers and school 
leaders widely accept that student learning — however it is measured — is a core 
measure upon which school success ought to be judged. In the past five years, 
more states decided to consider student growth an important component of 
teacher evaluations; 17 states now include growth as the most critical criterion in 
evaluations, up from only four states in 2009. 21 Another 18 states include growth 
measures as a “significant” criterion in teacher evaluations. 22 

There is evidence that teacher evaluations can serve as critical levers for teacher 
growth if they are tied to support and professional development. In Washington, 
D.C., for example, teacher evaluations have helped improve teacher quality 
because they were paired with an intensive support and development network. 23 
Connecticut’s System for Educator Evaluation and Development, or SEED, 
focuses on providing teachers with detailed, constructive feedback and custom, 
professional development opportunities, in addition to multiple chances to fre- 
quently discuss teaching practice with evaluators. 24 In Denver, the Framework for 
Effective Teaching helps teachers identify areas of strength and growth through 
feedback conversations, coaching cycles, and profession learning sessions, all 
before year-end evaluations. 25 

New evaluation systems can focus on growth for all teachers, as all teachers can 
benefit from feedback that helps them improve their performance. 26 

At the same time, these policies have also generated significant political backlash. 
One reason for this is that some states and districts have put in place new assess- 
ments in previously nontested — and even performance based — subjects, such as 
art and physical education, in order to meet the requirement that teacher evaluation 
ratings be informed by student growth. 27 In some communities, educators have 
come to see teacher evaluations as tools for discipline instead of growth, causing 
teachers to feel that they are not trusted by school districts and state legislators. 28 
In addition, in many communities, most teachers still do not receive any specific 
feedback in their evaluations regarding how to improve their performance. 29 
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The reauthorization of the Elementary and Secondary Education Act — now 
called the Every Student Succeeds Act, or ESSA — in December 2015, gives states 
complete autonomy over their teacher evaluation systems. 30 After state NCLB 
waivers expire in August 2016, states and districts will no longer be required to 
have teacher evaluation systems in place. Although Teacher and School Leader 
Incentive Program grants — formerly known as Teacher Incentive Fund grants — 
will continue to support states that implement performance-based compensation 
systems that are rooted in student achievement and multiple classroom observa- 
tions, federal funds will not be tied to federal teacher evaluation requirements. 31 
ESSA also authorizes, but does not require, the use of federal funds to develop 
mechanisms for effectively recruiting and retaining teachers and to create “human 
capital management systems” for teachers, which a local education agency can use 
to make personnel decisions. 32 

Over the next few years, states will be able to adapt their systems to work better 
within their unique contexts. As they do so, they may look at other states as models. 
Massachusetts’ system provides a framework that could prove illuminative to states. 
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The Massachusetts Educator 
Evaluation Framework: The basics 


Recognizing the limitations of Massachusetts’ teacher evaluation system in 2009, 
policymakers and educators sought to create a novel framework that prioritizes 
teacher growth. The new evaluation framework was designed to promote con- 
tinuous development, ensuring that educators have multiple opportunities and 
methods to maximize student learning. 


What was the state of educator evaluation 
in Massachusetts before 2010? 

Prior to 2010, educators, administrators, and policymakers alike felt that 
Massachusetts’ system of teacher evaluation could be improved. Notably, the 
previous district evaluation systems in Massachusetts: 

• Relied heavily on a small number of formal, announced observations 

• Often failed to differentiate between levels of effectiveness 

• Rarely identified excellence 

• Were inconsistent between districts 

• Did not promote self-reflection or continuous improvement 

• Were pro forma compliance exercises 

In August of 2010, Massachusetts Commissioner of Elementary and Secondary 
Education Mitchell Chester convened a 40-member task force to recommend 
a revised set of principles for educator evaluation. The Task Force on Educator 
Evaluation included a broad cross-section of stakeholders, including leadership 
of statewide organizations, active practitioners, higher education representatives, 
members of the business community, and local and national experts. 33 
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The task force concluded that educator evaluation practices in the commonwealth 
suffered from many of the shortcomings seen across the nation, including those 
listed above. The task force realized the need to create an evaluation framework 
that addressed these problems. It sought to create a framework that includes 
multiple measures of educator practice and honored the professional judgment 
of evaluators. The task force proposed the creation of an evaluation framework 
that includes three categories of evidence and a five-step evaluation cycle, which 
incorporates educator self-assessment. 


FIGURE 1 

The 5-step evaluation cycle 
of continuous learning 


Source: Massachusetts Department of Elementary 
and Secondary Education, "Quick Reference Guide: 

The 5-Step Cycle" (2014), available at 

http://www.doe.mass.edu/edeval/resources/QRG-5StepCycle.pdf. 



Teacher testimonial 

My evaluation experience has changed with the new framework. The 
new framework has been a more reflective process as compared to 
the previous evaluation system, which was more procedural — mean- 
ing: fill out the paperwork, answer the questions, and then put it in 
the drawer until it needs to be signed off. 

The new system gives a teacher specific language to follow, explain- 
ing what"proficient"or"exemplary" means. Therefore, it has impacted 


my practice, which ties to professional growth. As I go through the 
year I look at my practice and reflect on where I would fall on the 
rubric, asking myself, "What am I doing well and what do I need to 
work on?" I also look at planning, assessments, or communications as 
possible pieces of evidence or measurement of student learning. So, 
yes, it may go in a drawer for a time, but comes out much more often 
for reflection on practice as compared to the previous system. 

— Mary Stickley, teacher and union leader, Mashpee, Massachusetts 34 


8 Center for American Progress | Educator Evaluation 


How did Massachusetts successfully implement 
a new kind of Educator Evaluation Framework? 


The Task Force on Educator Evaluation’s recommendations were integral 
to the development of new educator evaluation regulations adopted by the 
Massachusetts Board ofElementary and Secondary Education in 2011, estab- 
lishing the Massachusetts Educator Evaluation Framework. 35 Importantly the 
task force represented a broad coalition of stakeholders who jointly agreed on 
these recommendations. The Massachusetts Teachers Association, for example, 
played a vital and leading role in developing the framework and working with the 
Massachusetts Department of Elementary and Secondary Education, orESE, and 
district superintendents to roll it out. 

The regulations included a staggered implementation process. Districts that 
participated in Race to the Top were required to implement systems aligned to 
the new framework in the 2012-13 school year with at least half of their educa- 
tors. In the 2013-14 school year, all remaining districts began implementation 
with at least half of their educators. By the 2014-15 school year, all Massachusetts 
educators were expected to be evaluated using systems aligned to the Framework. 
In addition, the regulations acknowledge the need for sufficient time to identify 
measures of student learning, growth, and achievement in nontested grades and 
subjects before districts would be required to incorporate student outcome data in 
the evaluation process. 


What were the framework's initial goals? 

The regulations, which apply to administrators and teachers throughout the state, 
were designed to bring consistency to all districts and to achieve five goals: 

1. Promote growth and development among leaders and teachers 

2. Prioritize student learning, using multiple measures of student growth and 
achievement 

3. Recognize excellence in teaching and leading 

4. Set a high bar for professional teacher status, or tenure 

5. Shorten timelines for improvement 
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What are the primary components 
of the Massachusetts framework? 


Massachusetts’ new framework, designed over the course of several years, allows 
for more local control, distinguishes between educator practice and educator 
impact, and ensures that teacher development is an essential component of the 
evaluation process. 


Local control 

In Massachusetts, procedures for conducting performance evaluations are subject 
to collective bargaining. Furthermore, education leaders in Massachusetts recog- 
nized that overly prescriptive regulations often make the mistake of imposing a 
one-size-fits-all model on a set of unique districts. As a result, the regulations that 
establish the Educator Evaluation Framework provide the principles of evaluation 
as determined by the Board of Elementary and Secondary Education but leave 
many details to the local level. 

To support districts in developing locally responsive systems, the ESE developed 
a model system that is consistent with the regulations and, with it, collective 
bargaining agreement language. This language was adopted or adapted with minor 
changes by most districts. 36 In sum, the regulations promulgated by ESE represent 
the non-negotiables in every district, and ESE’s model system and subsequent 
guidance serve as recommendations for local decisionmakers. 

The Massachusetts framework is designed to allow educator performance assess- 
ments to be made with a large amount of input from both educators and evalua- 
tors with a great deal of understanding of the context in which each educator is 
operating. In order to assess an educator’s performance, educators first conduct 
a self-assessment. The self-assessment is the first step of a 5-Step Cycle. During 
the first step, each educator also reviews data on his or her students’ learn- 
ing, as well as prior evaluations to assess his or her performance using a rubric 
anchored to the Standards and Indicators of Effective Practice described in the 
regulations. In step two, they use the result of their self-assessment to establish 
at least one student learning goal and one professional practice goal. Evaluators 
and educators discuss the self-assessment and proposed goals, working together 
to develop and implement an educator plan — step three — that fleshes out the 
next year or two of professional activities and supports that educators will use to 
meet their goals in steps four and five. 37 
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This experience is tailored to the individual needs of the educator. Only in cases in 
which an educator has been rated “Needs Improvement” or “Unsatisfactory” does 
the educator cede autonomy. In those instances, the evaluator plays a larger role in 
setting goals and developing the plan to target the areas that need improvement . 38 

Educators also play a lead role on the parallel side of the framework: measuring 
student impact. In most districts, teams of educators develop or identify the mea- 
sures they will use to assess student growth. At the end of each year, the educators 
and evaluators review student outcomes on the measures and reach an under- 
standing of whether each educator s students have demonstrated high, moderate, 
or low growth. After data have been collected from at least two measures in each of 
at least two years, the educators and evaluators discuss the body of evidence and 
the learning context in which the measures were administered, and the evaluators 
use their professional judgment to determine a rating of educator impact . 39 

By keeping decision-making as local as possible, Massachusetts sought to afford 
each district — and, ultimately, each school — opportunities to leverage the evalu- 
ation process to meet individual educator needs aligned to broader school and 
district priorities. 


Focus on growth 

The Massachusetts framework is focused on supporting educator growth and 
development. Using multiple measures of both student learning and educator 
practice, educators and evaluators work together to understand educator perfor- 
mance and develop plans to support his or her continuous improvement. Teachers 
who earn effective or highly effective ratings determine their own growth plans. 
Evaluators are responsible for providing educators with feedback and targeted 
support throughout the process. According to Colleen Mucha, principal of West 
Brookfield Elementary, part of the Quaboag Regional School District: 

The self-assessment piece drives teachers and administrators to thoughtfully 
identify areas for growth within their own practice. This has really transformed 
the mindset of many educators because they are much more open to the idea of 
continuous learning and improvement for themselves, which has made them 
more receptive to feedback . 40 
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The framework is not about compliance, conformity, or submitting the right forms 
with the right boxes checked. Instead, the evaluation framework is aimed at shift- 
ing the culture around teacher growth and development to encourage frank con- 
versations about teacher practice. This shift in culture took time. Massachusetts 
districts benefitted from a staggered rollout, both in terms of the number of 
educators evaluated — 50 percent of educators were evaluated in the first year, 100 
percent thereafter — and the types of evidence collected — measures of student 
learning and student and staff feedback were phased in over several years — as well 
as a framework that respects professional judgment and local flexibility. 


Trust and engagement 

In order for an evaluation system to be useful in promoting educator growth 
and development, teachers need to trust the system; they need to believe, for 
example, that the evaluation is an accurate portrayal of their practice. The tenets 
of the Massachusetts framework reflect the input of a diverse set of stakehold- 
ers synthesized by the ESE staff. Massachusetts has sought to build a world-class 
educator evaluation framework with the field, and not by imposing a mandate on 
the field. All of the guidance materials and policy decisions related to educator 
evaluation — and educator effectiveness more broadly — are vetted by members of 
the field before they are finalized. Adding stakeholder voices to the mix took more 
work and more time, but the process earned credibility for the framework and 
buy-in from educators and teacher advocacy groups. 41 

Because principals are the lynchpin of the framework in their roles as both 
evaluators and evaluatees, ESE’s Center for Educator Effectiveness established 
a Principal Advisory Cabinet, or PAC, in 201 1 to provide feedback on the ESE 
model system as it was being developed. 42 Teacher and superintendent advisory 
groups — known as TAC and SAC, respectively — followed soon after. 43 In the 
2014-15 school year, these advisors provided input on a host of educator effective- 
ness policies and initiatives, ranging from authoring a paper on building a school 
culture that supports teacher leadership, to assisting in the development of video- 
based educator evaluation training materials, to informing ESE’s decision to open 
up new flexibility for districts to develop systems for evaluating educator impact. 

In July 2014, ESE published a set of model student and staff surveys for districts 
to consider using. These would fulfill the regulatory requirement that student 
feedback be used in educator evaluation and staff feedback in administrator eval- 
uation. These surveys were developed by staff from ESE — including those with 
deep expertise in survey design and psychometrics — through a lengthy process. 
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They engaged extensively with stakeholders and consulted with more than 2,000 
educators who helped identify the most critical content to include in the surveys. 
It was essential that all items: 

• Were representative of the Standards and Indicators of Effective Practice they 
are designed to measure 

• Allowed students or staff to give an informed response 

• Provided information that educators can use to inform their practice 

• Used grade-level appropriate language 

In the 2014-15 school year, ESE selected four districts that have embraced edu- 
cator evaluation as a key lever for district and school improvement and profiled 
their efforts on video. The resulting video series depicts how four peer districts 
approached the task of creating educator evaluation systems and showed how such 
systems benefitted their schools. The video series was designed to help districts that 
might be lagging or struggling to launch an educator evaluation system. 44 The video 
series includes clips aligned to the 5-Step Cycle and clips designed to explore district 
systems that must be in place to glean the most from the process. 45 

Massachusetts found it useful to feature the work of districts that are finding 
success with implementation so other districts can learn from them. Connecting 
districts to each other builds more credibility and investment in the belief that this 
work is not only possible but also valuable. 


Juxtaposition of practice and impact 

One unique aspect of the Massachusetts framework is that educators receive 
feedback related to their practice separate from feedback related to their impact on 
student learning. The separation of these two ratings helps create a richer picture 
of educator effectiveness. 

In many other states, by contrast, educators receive one rating that is calculated 
using a formula or algorithm that includes a score based on observations of their 
practice and another based on their students’ performance on assessments. 46 
Sometimes educators understand where the rating comes from, but other times, 
they are not clear. As a result, educators and evaluators often lose the ability to col- 
laborate and have a conversation about why, for example, educators’ observation 
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scores were high but their students were not meeting expected growth rates or 
why educators’ observation scores were low but their students still demonstrated 
high growth rates. 47 These approaches underscore the performance rating but may 
miss the opportunity to really grow and develop educator skill and knowledge. 

The Massachusetts framework calls for every educator to receive two independent 
but linked ratings that focus on the critical intersection of educator practice and 
educator impact on student learning. Keeping the ratings separate offers educators 
and evaluators the chance to investigate any discrepancies between practice and 
impact. The summative performance rating measures practice and is anchored by 
the 5- Step Cycle that, unless educators are struggling, places them in a position to 
set the course for their own professional growth. 48 

The Student Impact Rating measures educator impact on student learning and simi- 
larly invests educators in the process. Teachers have opportunities to contribute to 
the identification or development of multiple measures of student learning, growth, 
or achievement. Teachers are also included in the process of determining parameters 
for high, moderate, and low student growth that are used to inform the ratings. 49 

The processes associated with both ratings are grounded in three main steps: 

1. Educators and evaluators collect evidence 

2. Evaluators analyze evidence from the educators 

3. Evaluators apply their professional judgment to the body of evidence and deter- 
mine a rating 


FIGURE 2 

The Massachusetts evaluation framework 

IMPACT RATING 



Low 

Moderate | High 

Exemplary 

One-year 
Self-Directed 
Growth Plan 

Two-year 

Self-Directed Growth Plan 

Proficient 

PERFORMANCE 

RATING 

improvement 

Directed Growth Plan 

Unsatisfactory 

Improvement Plan 


Source: Massachusetts Department of Elementary and Secondary Education, "Quick Reference Guide: Educator Plans" (2014), available at 
http://www.doe.mass.edu/edeval/resources/QRG-EducatorPlans.pdf. 
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While the two ratings are separate, they do intersect to determine the type and 
length of an educator’s plan. It is important to note that the summative per- 
formance rating must be based on evidence from multiple categories. It is also 
important to recognize that employment decisions are based on summative per- 
formance ratings. In other words: In the Massachusetts framework, an educator 
cannot be dismissed solely on the grounds of a low Student Impact Rating . 50 

Because districts have latitude to identify the measures used to determine educa- 
tor impact — even though measures of student growth do not exist in many grades 
and subjects — ESE urged districts not to link these ratings directly to employment 
decisions. ESE believes this decision has resulted in less teacher anxiety about the 
evaluation process than observed in many other states. That said, misconceptions 
about the consequences of the Student Impact Rating do persist, and the process 
of identifying and developing measures has been slower than anticipated, varied 
across districts, and unpopular with both teacher and administrator associations. 


Teacher testimonial 

Under the new evaluation system, I find that colleagues and I are constantly talking about 
the best practices described in the Teacher Evaluation Rubrics and finding ways to imple- 
ment these practices on a consistent basis. Between unannounced observations and 
evidence collection, teachers are making a genuine effort to improve their practice. I feel 
these efforts will have both an immediate and long-term impact on student learning. 

The evaluation system has me constantly analyzing my own practice, looking for ways 
that I can improve, grow, and develop as a teacher. Through self-evaluation and goal 
setting, I find there are lots of opportunities to recognize what works, doesn't work, and 
what can be improved. 

— Bryan Aries, Easton Middle School math and science teacher, 

North Easton, Massachusetts 51 
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What makes the Massachusetts 
framework different? 


The Massachusetts framework represents an alternative to many existing evalua- 
tion systems across the country. The framework uses test scores as a check on the 
rating instead of a component of the rating. It also prioritizes educator growth and 
improvement, applies to all employees in licensed positions, and allows for district 
flexibility in determining the specifics of the evaluation process. Massachusetts 
strives to use evaluation ratings to promote educator learning and positive changes 
to schools, as opposed to using ratings as a punitive metric of educator performance. 


Evaluator judgment: Front and center 

The Massachusetts Educator Evaluation Framework places evaluator judgment front 
and center — and educator growth and development is the primary focus. Educators 
and evaluators can juxtapose conclusions about educators’ practice with conclusions 
about their impact on student learning. As discussed in the previous section, within 
the Massachusetts framework, educators receive feedback related to their practice 
separate from feedback related to their impact on student learning. The ratings then 
intersect to determine the type and length of an educator’s plan. 

Although other models of educator evaluation, such as the Robert Marzano and 
Charlotte Danielson frameworks, helped inform the Department of Elementary 
and Secondary Education’s model rubrics, the Massachusetts framework focuses 
even more heavily on teacher growth. Proven educators are trusted to design their 
own growth plans, with the support of their evaluator. 


The framework is universal 

The Massachusetts framework does not apply only to teachers; it is used by all 
employees in positions that require a license. School nurses, guidance counselors, 
psychologists, principals, curriculum leaders, and superintendents, among oth- 
ers, are participating in the same process. This broad application has largely spared 
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districts from an us verses them set of challenges and has limited the likelihood that 
teachers would find the framework punitive. Many districts deliberately included 
all administrators in the first wave of their rollout plans to model continuous 
improvement for their educators. 

The task force initially observed that before the new regulations, there was little 
consistency in the expectations at the center of the evaluation process between 
districts. The Massachusetts framework addresses this problem by establishing 
four Standards and Indicators of Effective Teaching Practice and four Standards 
and Indicators of Effective Administrative Leadership Practice, which are all mea- 
sured in the evaluation of teachers, other service providers, and administrators . 52 
To support these standards, ESE released a set of four model performance rubrics 
that have been almost universally adopted by districts . 53 

The teacher and specialized instructional support personnel rubrics are aligned to 
the teaching practice standards, and the school-level administrator and superin- 
tendent rubrics to the administrative leadership practice standards. These com- 
mon standards and rubrics have helped Massachusetts develop a shared language 
across the commonwealth to describe practice and support the idea that all 
employees are working toward a singular goal: improving student achievement. 

ESE has received feedback from some professional organizations that believe 
specialized support personnel roles maybe too particular to fall under one of the 
four model rubrics. However, ESE has maintained the perspective that develop- 
ing additional role-specific rubrics would serve only to divide the field and foster 
isolation. Instead, ESE collaborated with these organizations to support their 
development of resources that help their members map their core responsibilities 
to the most appropriate model rubric . 54 

One consequence of an educator evaluation system that is over-reliant on the use 
of state standardized assessment data is that noncore academic teachers are often 
an afterthought. Their impact on student learning is frequently measured using 
student learning objectives, which are typically less structured and objective than 
common measures, or with schoolwide value-added data that reflect the English/ 
language arts and math outcomes of students whom the educator may not even 
teach. The Massachusetts framework, with its use of district-determined measures 
and the role of professional judgment in determining educator impact, mitigates 
this sidelining of noncore educators. 
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Principal testimonial 


The framework has helped me to become a better administrator because it requires you 
stay focused on the improvement of teaching and learning. It can be overwhelming 
sometimes. There are so many demands on a school administrator. As an administra- 
tor, I need to model the process of continuous improvement in a concrete manner. I 
share my own goals with my staff, I share some of my reflections when I identify an area 
where I feel I can do better, and I try to seek feedback from my staff on initiatives to let 
them see that I too am a part of the continuous cycle of improvement. 

At first, the framework is daunting. However, given time, patience, and a belief that we 
can always get better, it provides us with a roadmap to help stay on course and to keep 
the important things in our work — the important things. 

— Beth Chamberland, principal of Bryn Mawr Elementary, Auburn, Massachusetts 55 


Flexibility at the district level 

The Massachusetts framework attempts to honor the professionalism of evalua- 
tors by ensuring that multiple evidentiary components are available while leaving 
summative judgments in evaluators’ hands. ESE sets expectations about the types 
of information that should inform evaluations and defines the major process com- 
ponents. However, determining ratings and the process that must be used to make 
the ratings is left to local districts in consultation with teachers. 

According to Heather Peske, the associate commissioner for educator effective- 
ness at ESE, the evaluation framework is: 

statewide versus "choose your own adventure" at the local level: were trying to 
strike the right balance. Our colleagues in the field have largely embraced the 
opportunity to build evaluation systems that meet the goals of the state frame- 
work, while staying true to local priorities and needs. They are also always will- 
ing to let us know when they believe we at ESE have taken a step too far and to 
work with us to find a way forward . 56 

Most other state evaluation systems have taken a centrally mandated route. The 
Massachusetts framework, by contrast, provides flexibility to districts to align 
local evaluation systems to existing structures and culture. 
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Focus on human capital 


Rolling out the Massachusetts Educator Evaluation Framework has challenged ESE 
to enter previously uncharted territory for a state agency The creation and imple- 
mentation of the Massachusetts Framework required staff with specialized skill sets. 
Because ESE has focused on developing novel strategies to ground evaluations in a 
professional growth framework, it sought creative employees with highly specialized 
data skills and who are capable of communicating technical details to a broad audi- 
ence. ESE’s internal human capital strategy has three components: 

1. Recruit and retain high performers who are collaborative and enjoy creating 
new programs from scratch 

2. Provide ample opportunity for team members to think, strategize, and design 
together 

3. Unleash team member innovations by aligning staff with work that is of interest 
to them and granting significant autonomy in exchange for accountability 

ESE staff members — who were valued for their presentation skills and willing- 
ness to take risks — created a host of tools, including examples, for using common 
measures in educator evaluation, and communicated these strategies to schools 
and school districts across the state. 57 


Using the process to promote positive change 

With Massachusetts districts now in either year three or year four of imple- 
mentation, ESE is making a concerted effort to shift the lens to focus on how 
educator evaluation can promote positive change in schools and districts. For 
example, the process can help school and district leaders identify high-perform- 
ers who are ideal for teacher leadership roles — such as mentors, coaches, and 
professional development providers. It can also be used to identify individual 
professional development needs and promote specific instructional priorities 
aligned to school and district goals. 

In the 2014-15 school year, ESE launched a new approach to guidance develop- 
ment by establishing two Professional Learning Networks, or PLNs, around two 
key goals for educator evaluation: sustaining meaningful teacher leadership oppor- 
tunities and supporting evaluators to deliver high-quality feedback to educators. 
These were topics about which districts told ESE they wanted more support and 
ideas. The PLNs were comprised of small groups of 8 to 10 districts that applied 
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to participate and were accepted based on their demonstrated commitment to the 
topic. They were expected to commit time and resources to building or refining a 
project related to the topic area. The PLNs engaged in a year-long effort with each 
other and ESE to explore the topics and test strategies in their local contexts. 58 

ESE’s job was to support and chronicle the work done by the PLNs. In May of 
2015, both groups presented their work at ESE’s annual spring conference; over 
the summer, ESE posted a suite of new resources for the field-at-large to benefit 
from their learning. Managing the PLNs was time-intensive — including monthly 
planning calls with each district, several face-to-face meetings of each group, and 
targeted assistance and site visits as needed — but the end results reflect guidance 
that is largely already field-tested, adding to its credibility. In 2015-16, ESE has 
added a PLN on Educational Equity that is testing some of the strategies articu- 
lated in the Massachusetts Equity Plan. Each state is required to identify a plan 
to eliminate the inequities in student access to excellent teachers and principals. 
Additionally, ESE is combining the themes from last year’s PLNs into a single 
PLN on Distributed Leadership. 59 

Another example of a project designed to illustrate how the educator evalua- 
tion process can be used to promote positive instructional shifts is the Educator 
Effectiveness Guidebook for Inclusive Practice. In the 2014-15 school year, a report 
by Dr. Thomas Hehir and Associates found that, in Massachusetts, students with 
learning or communication disabilities who are educated full time in general educa- 
tion settings are nearly five times more likely to graduate high school in four years 
or fewer than are similar students in substantially separate placements. 60 Seizing on 
this finding, ESE staff recognized the potential of leveraging the educator evalu- 
ation process to promote inclusive practice in the areas of Universal Design for 
Learning and Positive Behavior Intervention and Supports. 61 ESE enlisted a cadre 
of teachers and administrators to write content, develop tools, and conduct mini 
field-tests, resulting in the guidebook. In the 2015-16 school year, ESE identified 
125 educators who have committed to testing the Guidebook tools and provide 
feedback as ESE conducts a comprehensive review in the summer of 20 16. 62 


Proficient as a rigorous, high bar 

From the very beginning, ESE communicated that “Proficient” represents a rigorous 
high bar and that “Exemplary” should be reserved for truly the highest performing 
educators. For example, the description of “Exemplary” in the model system states, 
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“A rating of Exemplary is reserved for performance on an Indicator or Standard that 
is of such a high level that it could serve as a model. Few educators are expected to 
earn Exemplary ratings on more than a handful of Indicators.” 63 

All educators are expected to meet a rigorous level of “Proficient” performance and 
the vast majority do. Of the 79,418 educators evaluated in the 2014-15 school year, 
only 9.6 percent were rated “Exemplary,” while 86 percent were rated “Proficient.” 
ESE is paying close attention to the rating distribution in order to understand 
whether the process enables districts to identify where teachers need support for 
improvement and to recognize and retain truly outstanding educators. 64 

ESE has also studied the relationship between ratings and gender, race/ ethnicity, 
new teacher status, and median student growth percentiles from state assessments. 
One of its primary findings from both the 2012-13 and 2014-15 school years was 
that teachers who were rated “Exemplary” in the summative performance rat- 
ing were more likely than other teachers to have achieved high student academic 
growth, as measured by median student growth percentiles from state assess- 
ments. Teachers rated “Needs Improvement” or “Unsatisfactory” were more likely 
than other teachers to have produced low student academic growth. 65 

In a related analysis, ESE found that teachers who were rated “Proficient” and 
“Exemplary” one year are much more likely to teach the following year than those 
rated “Needs Improvement” and “Unsatisfactory.” 66 This finding is evidence that 
the framework is delivering on the goal of shortening timelines for improvement 
and supports the notion that districts are identifying high performers and retain- 
ing them at high rates. 


FIGURE 3 

Teachers who received ratings in 2014: Were they still teaching in 2015? 


Exemplary 
Proficient 
Needs improvement 
Unsatisfactory 


46% 


91% 


Source: Authors received materials from the Massachusetts Department of Elementary and Secondary Education on January 26, 201 6. 
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Principal testimonial 


The current educator evaluation model differs from past models in its sustained effort 
on improving teacher effectiveness, teacher reflection, a focus on student learning, 
and constant growth. While in the past, professional conversations have occurred in 
pockets, the new educator evaluation model provides a structure and clear expectation 
for all staff to engage in professional conversation, reflection, and growth. The shift in 
culture has been powerful, and all educators — of all levels and titles — are now consis- 
tently expected to engage in a constant growth cycle. 

The new educator evaluation system has had a large impact on educator growth and 
development. Educators have increased their collaboration around team goals — which 
has increased communication and collaboration in our school — and all educators are 
expected to focus on growing their own practice. This has become a very collaborative 
process, opening lines of communication between educator and evaluator, focusing on 
student learning, and reflecting about areas of student need and also professional need. 

— Steve Guditus, principal of Manchester Essex Regional Middle School, 

Manchester-by-the-Sea, Massachusetts 67 
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Lessons learned 


Other states can take several lessons from Massachusetts’ experience in creating 
the Framework. Many of the hurdles that Massachusetts faced could have been 
avoided with more strategic planning, as well as a greater emphasis on educator 
feedback and providing more local support during rollout. 


1. Titles matter 

The Framework would likely generate more support if it were called the 
Massachusetts Educator Growth and Development Framework, instead of 
singling out and incorporating the word evaluation in the title. The Department 
of Elementary and Secondary Education’s messaging and guidance have been 
focused on using the process to support and coach educators to reflect and 
improve. The evaluation process results in dismissal for only a small minority of 
educators. Framing this work as evaluation has slowed educator buy-in and belied 
the framework’s true intent: to ensure that all students have access to excellent 
educators — a feat most effectively accomplished by strengthening the existing 
workforce, rather than replacing it. 


2. Set implementation dates in order to streamline negotiations 

The Educator Evaluation Framework might have been more smoothly imple- 
mented if laws or regulations bounded the time for collective bargaining. 
Moreover, Massachusetts could have emphasized that parties that failed to reach 
an agreement by a certain date would be required to use the model system. Such 
requirements would have encouraged parties to adhere to clear timelines. Districts 
frequently report to ESE that implementation delays have been the result of slow 
or stalled negotiations — a problem that has hindered the rollout of the Student 
Impact Rating, in particular. 


23 Center for American Progress | Educator Evaluation 


3. Regional training and support are important 


ESE conducted regionally based Getting Started workshops, which were well- 
received but limited due to ESE’s capacity. 68 These efforts might have been more 
effective if teams of educators — including both teachers and administrators — 
were targeted, rather than just administrative teams. A more robust regional sup- 
port network to strengthen local dissemination of professional development and 
guidance could improve training and support efforts. 


4. Evaluation and curriculum should be aligned 

Expectations for teacher and administrator practice should be inextricably linked 
to expectations for curriculum and instruction. ESE could have done more to ini- 
tially focus on this connection. As a result, teachers, administrators, and advocates 
have shared that ESE’s work in these areas felt like a series of disparate initiatives 
that distracted from, rather than buoyed, each other. ESE has made strides to 
improve model initiative integration. 

The Student Impact side of the framework relies heavily on district-determined 
measures. Identifying or developing Student Impact measures has proven to be a 
major undertaking in many districts — especially in traditionally nontested grades 
and subjects. 69 Districts that are still working to implement the framework would 
benefit from a stronger collection of example district-determined measures that 
are aligned to the Massachusetts Curriculum Frameworks. 

ESE continues to focus on strengthening the connections between educator 
evaluation and curriculum and instruction, both within the agency and at the 
district level. Given that the model performance rubrics assess elements that are 
applicable across content areas, ESE is working to develop resources that sup- 
port educator growth in teaching the content of the Massachusetts Curriculum 
Frameworks. For example, ESE recently published “What to Look For” — a series 
of observation guides that are available for English/language arts, math, and sci- 
ence in grades K-8 — which connects the Standards of Effective Practice assessed 
in educator evaluation with the content standards. 70 
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5. Do not overlook a communications strategy 


Race to the Top funding allowed ESE to bring on a communications firm to help 
tailor communication to appropriate audiences, employ video messages, format 
guidance materials to make them more accessible, reduce jargon, and help the 
field understand how ESE’s initiatives are connected. 

Repeating consistent messages to varied audiences is essential. When rollout 
first began, ESE relied too much on targeting superintendents and expecting 
trickle-down communication. Such a strategy was not effective, and ESE has since 
adjusted course. For example, ESE introduced an Educator Evaluation newsletter 
in February 2013 that was initially targeted at district leaders. 71 Over time, ESE 
has broadened the content to appeal to a more diverse audience and expanded 
topic areas to highlight connections between educator evaluation and other 
educator effectiveness or agency initiatives. ESE has also launched a new direct- 
to-teachers communication called, Teachers Top 3 from ESE. 72 Most importantly, 
ESE has learned from its Teacher Advisory Cabinet about the importance of 
“leading with the why” when communicating with the field. 73 The purpose behind 
the project or policy is just as important as the approach. 


6. Implement impartial feedback loops 

ESE contracted with an external research team to study the Educator Evaluation 
Framework during the first three years of implementation. 74 The researchers 
conducted statewide surveys of teachers and administrators, ran focus groups 
and interviews, and produced case study reports on topics of interest. According 
to Carrie Conaway, the associate commissioner for planning and research at 
ESE, “Each year, we learned something new and adjusted course accordingly. For 
example, after the first year of the study, we learned that most educators under- 
stood the 5- Step Cycle, which meant the program office could focus on new areas 
of work in the second year.” 75 Most importantly, ESE was provided with regular 
research briefs that they used to understand successes and challenges in near real- 
time, as opposed to waiting for summative annual reports. 76 The research findings 
have been invaluable in setting strategic priorities and governing the development 
of new resources. ESE has communicated the findings to the field, such as using 
the findings in memos sent to superintendents. 77 
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Teacher testimonial 


Throughout that past few years, evaluations have guided which professional develop- 
ments I participate in for the school year. For example, one of the goals for my evalu- 
ation cycle, which began two years ago, was to identify and implement close reading 
strategies for fiction and informational texts in my eighth grade ELA classes. My district 
paid for me to attend a workshop on close reading strategies, and I now regularly utilize 
the specific strategies I learned from the workshop in order to increase students' com- 
prehension of the texts we read in class. 

— Terri Dsida, Norwell Middle School English language arts teacher, 

Norwell, Massachusetts 78 
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Where do evaluations 
go from here? 


There is more work to be done to fully implement the Massachusetts framework, 
particularly as it relates to helping districts leverage the evaluation process as a 
means to improve school quality — such as by retaining high performers by pro- 
viding teacher leadership opportunities, targeting professional development offer- 
ings to specific educator needs, and better understanding the educator practices 
that produce the most student growth. 

As the 2015 Every Student Succeeds Act places more accountability for educator 
development and performance on states and phases out No Child Left Behind 
waivers, Massachusetts has already begun discussing the consequences of the 
new law. 79 For example, the Department of Elementary and Secondary Education 
is interested in taking a close look at the model performance rubrics. 80 Are they 
rigorous and useful tools for evaluating educator practice? Are certain indica- 
tors or elements of the rubrics more predictive of overall performance? Are there 
redundancies that can be eliminated? Do the descriptions of practice help evalua- 
tors make meaningful distinctions? 

The summative performance rating has been met with stakeholder reactions that 
range from acceptance to excitement. The Student Impact Rating is a different 
story. ESE knew from the outset that the use of measures of student learning, 
growth, and achievement to determine a rating of educator impact would be chal- 
lenging. Nevertheless, the department sees the Student Impact Rating as essential 
to an Educator Evaluation Framework that keeps student learning at the center of 
the process. 81 By keeping the summative performance and Student Impact Ratings 
separate, Massachusetts has taken a balanced approach. The Student Impact 
Rating is a check on the system, ensuring that educators do not feel that test scores 
wholly determine their effectiveness. The framework keeps student growth as a 
critical goal, but the focus is on other indicators of instructional effectiveness that 
are more connected to practice. 
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The time and effort necessary to develop or identify measures — especially in 
traditionally nontested grades and subjects — has proven considerable in many 
districts. While many districts have robust assessments in place in English/ 
language arts and math, the same is not true for subjects such as science, social 
studies, physical education, and the arts. The work underway to develop common 
measures for the Student Impact Rating is helping all educators improve their 
assessment literacy and fostering common expectations that educators collect, 
analyze, reflect, and act on student assessment data as part of their craft. 


Data collection and feedback 

In Massachusetts, most educator evaluation data are maintained locally at the 
district level. ESE collects very few data points from districts, which makes 
understanding the quality of implementation a challenge. 82 Adjusting policy to 
collect more data will likely be met with resistance from the field. 83 ESE will have 
to determine how to collect better data about the quality of implementation, 
while maintaining support from stakeholders and not over-burdening districts. 
The research study described above, which provided ESE with a great deal of data 
about implementation over the first three years, was funded using Race to the Top 
dollars and ended when those funds were exhausted. ESE is still exploring ways to 
fill the data void that was created when the study ended. 84 

ESE still must determine how districts can best distribute leadership responsibili- 
ties in order to ensure that educators receive relevant and meaningful feedback, 
while providing teacher leadership opportunities and balancing evaluator work- 
load. ESE launched a Distributed Leadership Professional Learning Network in 
the 2015-16 school year in order to explore this question together with districts. 


Teacher preparation and professional learning 

ESE is continuing to strengthen the connections between educator evaluation, 
educator preparation, licensure, induction and mentoring, and professional devel- 
opment. ESE has endeavored to create a seamless educator-career continuum by 
connecting expectations for educator preparation to the language of the standards 
used in evaluation. 85 As part of a multifaceted effort to better prepare teacher 
candidates to meet the demands of 21st century classrooms, ESE first aligned the 
standards for preservice with the Standards of Effective Teaching Practice. 86 ESE is 
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also piloting a new performance assessment for teacher candidates that is aligned 
to the Educator Evaluation Framework and will become a graduation requirement 
for all candidates who attend Massachusetts preparation programs beginning in 
the 2016-17 school year. 87 


Ensuring variability within teacher ratings 

By the 2014-15 school year, all Massachusetts educators were evaluated using 
systems that were aligned to the framework. However, the effect of Massachusetts’ 
Educator Evaluation Framework on improving student achievement has not yet 
been measured. The state is studying the results from 2014-15 for evidence of rat- 
ing inflation. Ratings tell only part of the story. In addition, ESE plans to continue 
to collect information on the success of district implementation and how districts 
are ensuring that teachers are receiving meaningful feedback. This information 
will be gathered through the District Review Process conducted by ESE’s Center 
for District and School Accountability, as well as a new statewide survey coordi- 
nated by the Office of Planning and Research. 88 


Teacher testimonial 

As a new teacher, the evaluation system has acted as a guide of what I should be doing 
in my classes. The key part of this evaluation for me is the reflection piece. It requires 
me to sit down and actually think about what I do to prepare for class and in the 
classroom. I think it has helped me identify areas of weakness and an opportunity to 
strengthen them. 

— Alyssa Miller, Somerville High School physics and engineering teacher, 

Somerville, Massachusetts 39 
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Conclusion 


The Massachusetts Educator Evaluation Framework supports educator growth 
and development by providing significant autonomy to educators who have 
demonstrated consistently high performance and targeting support to those who 
have not. The framework allows educators, evaluators, and district leaders to juxta- 
pose conclusions about educators’ practice with conclusions about their effect on 
student learning in order to investigate discrepancies between educator actions 
and student outcomes. 

Massachusetts chose to trust evaluators, who work closely with the educators, 
in order to determine the ratings. The state education board’s role is to set the 
principles of evaluation, and the department’s role is to provide support and guid- 
ance to local education agencies in order to determine details that will be effective 
in local contexts. The framework provides flexibility to districts to align the local 
evaluation system to existing structures and culture. Similarly, local education 
agencies should empower well-trained evaluators to identify the specific feedback 
and supports that educators need to improve, and evaluators should trust educa- 
tors who have demonstrated high performance to set their own courses. 

The Massachusetts Educator Evaluation Framework is not a panacea for improv- 
ing educator performance and student outcomes. Implementation has yielded 
as many questions as answers about how to provide educators with high-quality 
feedback at scale and with consistency, particularly as it relates to feedback about 
educators’ impact on student learning. Massachusetts must continue to use the 
system to track and analyze data on educator improvement in order to ensure that 
the system is meeting its stated goal of driving continuous growth and improve- 
ment in teacher practice — especially given recent research findings that teach- 
ers plateau in their growth after time . 90 Rigorous evaluation and a willingness to 
course correct when needed will be essential to success. 
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